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Three Ways of X earning Morg - From Follow Throagh^ 
Secondax^^nal^ysls of Ext:ant Data, Gompiiatton and Analysis 
of Follow-tJp Data, and Completely New Studies 



This paper has been piriapared for the National Institute of Education as 
part of thexr agreement with the Office of Elementary and Secondary Education 
to participate in new Follow Through (FT) research and pilot project activities. 
The paper's objective is to address three questions related to FT's potential 
for generating hew knowledge: 

• What more can we learn from FT? through secondary 
analysis of extant data? What FT-re:lated_ data bases 
exiSt at the national, sponsor, and site levels? 

• What can we learn from compiling and analyzing 
follow-up data on children who participated in FT? 

• What can we learn from new studies of sponsors and 
Sites m rr's current framework? 

Implicit in the title o£ this paper is the conviction that there 4^ more 

to be learned from FT. This belief stems from FT's history as well as from the 

fact that FT how has a dual Tocns. New regulations mandate that 80 percent or 

PT's funds be used for the provision of service while the remaining 20 percent 

are to be used for knowledge production (Wholey 1979). The regulations thus 

reinforce the viSion of FT as a federally funded education program containing a - 




buxlt-m research component. There are few enough programs with a mandate to 
do educational research and knowledge production < and enough of a need for 
research, so that by any definition there is a substantial amount to be learned 
from FT. ' 

The national FT evaluation (Stebbins, St. Pierre, Proper, Anderson and 
Cerva 1977) showed that ixi terms of certain outcomes, FT children in some models 
performed better than comparisbri children, some performed about the same, and 
some did worse. These results are not particularly upsetting if we see FT as a 
research program. From this viewpoint, the national evaluation was not an 
evaluation of FT (which would have assessed the adequacy of FT as a vehicle for 
providing a testing ground for the exaitiihatibh of the effects of model sponsors), 
but was a part of FT. 

Regardless of the outcomes of the national evaluation, FT continues to 
have worth as a service and knowledge production program designed to develop, 
test, and disseminate alternative models of education. This statement ignores 
the debate over claims of model effectiveness and is based on the opinion that 
FT addresses an important problem. It is the largest and most sophisticated 
vehicle that the federal gbvernihent has for testing the effects of alternative 
educational models; it fuhctibhis to provide funds for curriculum research and 
development under field conditions; and it has already generated a wealth of Icnow 
ledge about primary school education, the implementation of external ly-deve^ oped 
programs in the public schools, and evaluation methods fbr field-based programs. 
Because of this history, and the prospect for continued research, development, 
testing, and disseminatibri of educational practices, FT has probably come closer 
to being a "reform as an experiment" (Campbell 1969) and to Campbell's (1971) 
vision of the "experimenting society" than any other federal prberam. 



2 



4 



There certainiy is room for disagreement with this opinxon. tn fact, there 
are two widely held vxews that lead to very different conclusions about whether 
or hot we can learn more froiTi FT. One of these is held by many FT practitioners, 
administrators and Cbngf essidnal suppdirters-'-that FT is a service prbgraiti and 
only a service program. In this vxew resources allocated to FT research are 
resources not allocated to service and by definition are ill-spent. Clearly, a 
person holding this view does not see FT as a prime vehicle for generating new 
knowledge. The second opposing view/ held by many in the current admiriistf atibh , 
xs that FT IS simply an expensive version of Title t and should be phased out. 

The fact that persons with these three major views (FT as a program capable 
of supporting both service and important research, FT solely as a service p>rbgram, 
and FT as a prbgram to be phased but) have tb interact ta the process of planning 
FT research has lead to great difficulty in organizing, planning/ and carrying out 
any sort of research. This point will be addressed again later. 

Thxs paper xs written under the assumption that the real c5uestioh for persons 
holding the first vxewpbirit articulated abbve is nbt whether we can learn mbre 
frbm FT xn its current framework, but how much in the way of scarce resources 
should be invested in gaining what kinds of knowledge. The paper discussas two 
ways of mxnimizing such an investment through the use of existing data oh 
children who have already completed the FT prbgram. In addition/ it reviews and 
cbmmehts bn some of the suggest xbns for ':ew FT research studies that have been 
proposed in the four years since completion of the national evaluation. 



1 .0 What: More Can We Learn from FT Through Secondary Analysis of Extant: Data? 

1 . 1 Motzvatibn for St:udxes Based on Extant Data 

There are important questions to be cuiswered about and by FT- Further/ 
relevant data sets exist or can be constructed from school records to address 
some of these questions. While the constraint of relying on accessible, existing 
data limxts the range of questions that can be answered, the advantages of 
performing secondary analysis (Cook 1974) are considerable. Most important are 
that studies based oh extant data will be less expensive and more timely than 
those involving substantial new data collection. 

Secondary analysis requires minimal resources since costs associated with 
designing the original evaluation, develbpirig instruments, collecting the data, 
and building the data base have all been met by others (^Clfuch and Reis 1980). 
The secondary analyst must bear only the costs of data access, analysis, and 
report writing. Further, because it shortcuts the startup, planning, data 
collection, and data base construction periods of an evaliiatibh , secondary 
analysis of extant data can yield information in a fraction of the time that it 
would take to mount a new study. 

The major drawback to secondary analysis is that the questions that can be 
addressed are limited by the nature of the extant data. It is often the case 
that the secondary analyst has different or more detailed questions than were 
asked in the primary evaltaation- Variables that are key to the secondary 
analyst may be missing from the data base. It should also be recognized that 
for some questions (e.g. those that are most easily addressed through case 
studies or other qualitative methods) secondary analysis may be infeasible or 
less cost effective than the collection of completely new data. For many such 
questions the data simply do not exist, or if they do, the circumstances of 
their collection are so unique that the data have little applicability to other 
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situations. However, assuming data exist to support research on impbrtarit 
questions > secondary analysis is an efficient and tiiiiely mechanising 



Many data sets related to various aspects of FT are available including the 
national evaluation data base, several sponsor and site maintained data bases, 
and data sets built through other federally f\ihded FT research projects. 

1.2,1 National Evaliiat ion^J^ta-Base 

Perhaps the most obvious, though not necessarily the best, extant resource 
for learning more from FT is the data base resulting from the ten-year national 
FT evaixjation. Data on over 20 FT sponsors that implemented their educational 
models in more than 200 sites nationwide are contained in this massive data set. 
The data base is organized into four cohorts which comprise FT and NFT children 
from ail sponsors and sites. within each cohort the data base is hierarchically 
organized by sponsor, site within sponsor, school within site, class within 
school, and child within class. Contained within these sets are various types 
of data including child level demographic data, child level test scores > child 
level parent interview data, and classroom level teacher and teacher aide data. 
For some children, only demographic data are present; for many, the other types 
of data are present for up to four years (grades K-3 ) . Tables 1, 2 and 3 are 
adapted from Goodrich and St. Pierre (1979) and give some details on the SHI 
data base. 

Middleton and Durgin (1978) have documented the national evaluation data 
base as constructed by SRI International. However, the national evaluation 
data set is hd longer maintained by SRI. A copy has been submitted to the 
National Archives and to this writer's knowledge at least one attemp-t to 




EKLC 



5 



Table 1 

Demographic Cata on the- S^I- Dat^a -Ba^ * 





Cohort 


I 


II 


III 


IV 






cv 


EF 


-^K 1 


EF 


— EK 


EF - 


EK 




— Stream — 














variable 


Grade 


1,2.3 


1.2,3 


K, 1,2 


3 


1.2,3 


K, 1, 2,3 


. 1.2,3 


K.1,2 


Project 






X 


X 


X 


X 


X 


X 


X 


School 




X 


X 


X 


X 


X 


X 


X 


X 


FT/NFT designator 




X 


X 


X 


X 


X 


X 


X 


X 


Grade 




X 


X 


X 


X 


X 








Child bxrthdate 




X 


X 


X 


X 


X 


X 


X 


X 


sex 




X 


X 


X 


X 


X 


X 


X 


X 


Ethnxcity 




X 


X 


X 


X 


X 


X 


X 


X 


First language spoken 
in the Home 




X 


■■ 

X 




X 


X 


X 


X 


Second language spoken 
in tne nome 




X 


X 


X 


X 


X 


)C 


X 


Head Start flag 








X 


X 


X 








Head Start equivalent 


flag 








X 










Months Head Start of 
■aquj-valf^nt experience 


X 


X 


X 


X 


X 








Months Head Start experience 












X 


X 




Mohrhs equivalent experience 












X 


X 




Days absent during year 






_ 

X 


_ 

X 


X 


X 


X 


X 


Child eligible for FT 


services 








X 




X 


X 


X 


Classroom service avaxla&le 








X 




X 


X 




Lunch service available 








X 




X 


X 


X 


Medical, dental service 
ava ilable 








X 




- 

X 


X 


X 


Other •ser'x/ice available 








X 




X 


X 


X 


Date child entered class 








X 










Date child left class 










X 










Years of daca available 

oh child 












X 


X 


X 


Parent interview history 












X 


X 


X 


Child test history 














X 


X 




Child FT/NFT by year 














X 


X 


X 


Mon:ihs m FT prior to 
this year 














X 


X 


X 


Child entering grade 


















X 
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Table 2 
Test iJala on the SHI Data base" 





COUOtff 


I . 


11 . ._■ 


"■"III — . .._ 




mm 


£K- - - - 


ff- - - 






- 




EK - - - - 


EF - 




EK 




Mm 


K- 




-2- 


-- 




-\r- 




4- 


4- 


4- 2- 


^- 








- -- 




( - 








4 


— 








[— 


4- 




m 


Ffi9 




















P10 


















































POINT 


S'/O 




S'/l 


F71 


i'l2 




S]0 


r/o 


S?l 


S12 


S?l 


S72 


F72 


sr3 




F70 


SIl 


S?2 


F72 


S73 


F7I 


S72 


F72 


S73 


S7^ 


S7b 


F71 


S72 


F72 


S73 


37^1 


F72 


373 


37^ 


S75 


m\ 




X 


X 




— 




X 


x 

1 


X 




X 










X 


X 








X 


X 










X 


X 








X 








psi 










— 


— 


X 






— 


X 










X 


x 








X 












X 


















Ltt-iliii(K 


X 






— 






X 






— 


X 
X 








— 


X 


X 








X 




- ■ 


- 


— 




X 


















tci 

FACES 


X 
X 






— 


— 
— 


— 


X 






— 








X 


X 
























— 




— 






X 


X 


— 


— 


— 


X 


X 


X 


— 


X 










X 


X 








X 


— 


— 






— 


X 












— 




— 


KjcIiiL lIlTtUIS- 


X 


x 


X 






x 


X 


X 




x 










X 


X 




























_ _ 
















X 


X 




— . 


— 




X 


X 


— 














X 


— 














— 


— 














— 






SPONSOR JTM.. . 
(¥ElltiAL t ttUANT. ) 




X 


X 




— 






















X 


X 
































• — 
— 


— 


— 


sPoiisoH um 

(HISC.) 


— 


X 


X 


— 


— 


— 




X 


X 


— 














- 














— 


— 














HAT HllMtll 


— 


































— 








X 


X 




















X 






MAT i'lllMAIiy i 
















X 








X 


X 






















X 








X 


x 










X 




HAT I'lllHAKY It 








K 


X 








X 










X 








X 


X 












X 










X 










X 


HiiT miwm 












X 








X 










X 










X 












X 










X 










mm 








X 




X 








X 










X 






X 




)• 












X 










X 










COOItllSHlTII 








X 




X 








X 










X 






X 




X 












X 










X 










lAliS 

uUhI'IMI ts 








X 




X 








X 










X 






X 




X 












x 










x 


















X 




















X 












X 








X 








LUCUS CUNTIIU. 






















X 




















X 












X 
















PFVT 


























11 
















X 
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Table 3 
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h» ri* 


Nu Tfbl 
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hi KT 
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No TciiL 
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I'Otit- 
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liuic 






I'lL- 
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lo:ttt;iI 


iiuiitoii 




iii^iii 
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Icbttid 






IIIClll 


~~~ 




- 










~— 





























b44 








699 


3b4 




2UU 


179 


223 




■ -- 







301 




202 









b^J 


367 


230 


21!) 


fcyb 


3U3 


30d 


2U5 


550 


495 


2y4 


293 


50) 


505 






Aiuijii-i 


%6 


639 


341 


31? 


?d2 




354 


307 


745 


501 


4j0 


4l0 


7oy 


609 








bbd 


4J2 


270 


2i>0 




34^ 


2b6< 


234 


441 


354 






397 


377 


01 


71 




iii/ 


lUl 






nti 


1U9 






419 


301 


292 


203 


320 


305 










tJb 


2bl 


241 


772 


505 


2bi 


25U 


771 


503 


27fl 


236 


022 


749 


225 


136 






002 


G21 


559 


Duo 


G79 


369 


34 i 


59 


49 






223 


200 






I1i«jIi/^lo|K: 


ii>j 


222 


147 


131 


2iy 


106 


ii4 


lUU 


215 


210 






237 


222 


125 


45 


No I ill CJloiiiM 




335 


241 


230 


570 


421 


289 


239 


5&I 


447 






407 


342 


191 


1?4 


UjC 




23U 


222 


2UI 


30O 


172 


132 


92 


2o5 


231 






27a 


204 


239 


210 






2b 1 


254 


235 


172 


141 






yo 


07 






iiii 


70 






Futilliiiiii 


m 


129 


i23 


116 


239 


;!05 


54 


49 




















m 


4U4 


393 


3^7 


575 


J 29 


220 


iu3 


111'/ 


104 






22? 


21U 






llaiiipluti 


to 
















43 


31 






54 


31 
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177 


73 


12 


fil 


105 
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34 
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obtain the data through that irbute was successful. Abt Associates Inc. (AAI) 
maintains the analytic files used In the national evaluation on an ad hoc 
basis and has distributed copies to researchers upon request- 

1.2.2 FT Sponsor and Site Supported Data Sets 

The riatibrial evaluation data base includes many sites and sponsors with a 
common, but restricted, set of measures. It has breadth but lacks depth. On 
the other hand, internal evaluations done by several FT sponsors (c.f. Hodges 
and Sheehan 1978) cbmplement the national effort by adding depth in selected 
instances. Some sponsors developed their own instruments, others used standard 
tests not employed in the national evaluation. These efforts were idiosyncratic 
bt;*" contain information that is sometimes more relevant to the sponsors' objec- 
tives than the national data. 

While studying the utility of sporisbr and site maintained data sets for 
supporting research bh the Ibng-term effects of FT, Goodrich and St. Pierre 
(1979) surveyed all FT sponsors and several sites as to the existance and 
nature of relevant data sets. Table 4 summarizes the results of this, survey 
for some of the sponsors/sites having data sets most amenable to secbridary 
analysis . Several of these data sets could support research on the effects of 



FT during and subsequent to Zhe FT years. 

Sponsor/site data sets vary tremendously in quality. The best (i.e. some 
of those shown in Table 4) are characterized by several years bf longitudinal 
data oh FT and NFT children, well matched comparison groups, pretest and posttest 
data and many measures. Some contain measures specifically designed to tap 

*This survey was current as of early 1979. Since that time some sponsors 
have undertaken significant data base development arid any thorough examinatibri 
of sponsor data should include an update of the current status bf these data 
sets . 
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Table 4 



Summary of Selected Sponsor/Site Data Bases 



Data Beise 
Maihtaip. e d By - 



^n^^y??*! of Arizona/ 
Tucson Early Sducation 
Model 



Sites/Sponsors 
Included in 
Data Base 
Wichita, KS 



Ford hast Onxvefsity/ Atlanta, GA 

Ihterdepehdeht Learning New York City, NY 
Model 

Georgia State, University/ Natchitoches 

Parent Supported Parish, LA 
Diagnostic Model 



Mea s ures 



Ach levement t es ts , 
productive lan- 
guage test 



1-6 Achievement tests , 
problem solving, 

P^°h^c skills , 

many noncognitive 
measures 

1-6 Achievement tests, 
attendance 



Coffipartson 

Groups 

Randomized across 
treatment/control, 
other nohequiva- 
lent cbnipar isons 

well-inatched 
local cdniparisoh 
groups in Atlanta, 
f air__to poor in 
New York City 

Well-matched 

local comparison 
groups for three 
of six schools 



Estimated 
Lonoitudinal 
Sambl«^&i«e — 



ibbi per cohort 
foe several 
cohorts 



UOOO's in 
Atlanta, 100*s 
in New York 
City 



Low JpO's. 
across all 
cohorts 



Oniversicy Of Kansas/ 
Behavior AhstLysis 
Approach 



Oniverstty of Oregon/ 
Direct Instruct ion 
Model 



New York City, NY K-5 
Philadelphia, PA 
E>ortageville / MO 
Trenton^ NJ 
Kansas City, MO 
Louisville , KY 
Waukegan, XL 
Meridian, IL _ 
N. Cheyenne, (fT 
Hopi , At 

Flippm, AK _ K-6 
E. St .Louis , IL 
Sraithyille< TN 
Uvalde, TX 
Dayton, OH 
Tupelo, Mis 
Flint, MI 



Achievement tests , 
attendance , 
consumer satis^ 
faction 



Achievement tests, 
attendance 



well-matched 
local conparisbh 
groups in urban 
sites, poor or 
Comparisons 
in other sites 



Well^atched . 
local cahparisori 
groups in most 
sites 



100 's per 
cohort for 
several cohorts 



100*3 per 
cohort for 
tvo cohorts 



Southwest Educational 
Oevelopment Laboratory/ 
Bilingual/Bicultural 
Model 



Cutler-Orbsi , CA 
Los Angeles , CA 
San piego , TX 
St. Martin Parish, LA 
Ttilare , CA 



K-6 Achievement tests, 
Spanish reading 



Adequat fily 
matchec'i locztl 
comparison 
groups 



75-150 per 
site per 
cohort 



Philadelphia 
Public Schools 



Bank Street 
EDO 

Kansas 

North Carolina 
Phila* Process 
Parent Implem. 
SEDL 



1-6 Achievement tests, 
attendance, info, 
oh ccnipetihq 
treatments 



Cross-sponsor 
comparisons , 
well -matched 
local can par ison 
aroups up to 1975 



100's per 
sponsor per 
cohort 
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features of the sponsor's educational model. Fbr example, the Fordhaih data base 
cbhtairis measures of problem solving as well as rtiany other ribncdgnitive measures, 
while the Oregon and Kansas data sets are strong in tenns of academic achievement 
measures ; 

Some of the data sets are very strong in terms of design. For example, 
the data set supported by ah ALizoha site (the Wichita Public Schools) contains 
data from a classical experiment where Heed Start-eligible children were assigned 
randomly to TT and NFT groups across several cohorts . 

With a few nestable exceptions (e.g. Philadelphia, Fordhamj most sponsor and 
site data bases are not readily transportable. They terid to exist in bits and 
pieces arid would require a modest effort to pull together . Even some of the 
best data sets in terms of measures and design (e.g. Kansas, Oregon) exist in 
cross-sectional files that would require merging prior to longitudinal analysis. 

1.2.3 Other Research Based Data Sets 

While the hatiohal evaluation and the FT sponsors are the primary sources 
of extant data, some ongoing research studies will provide important data sets. 
First, System Development Corporation is conducting a study of parental involve- 
ment in four federal programs (FT, Title I, ESAA, and Bilingual Education). In 
the course of this study (which is riot yet completed) SDC researchers collected 
two types of data. A national survey was conducted to determine the nature and 
level of parent involvement activities in FT (as well as in the other federal 
programs noted above) . The data base from this survey will be available in the 
near future. The secorid type of data collected was iridepth case study data on a 
sample of 16 FT sites that were determined through the national survey to be 
active in terms of parental involvement. The data base for this study is not 



yet complete, but it wrii likely consist of hard copy case study materials. 
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Second^ the Network for Innovative Schocis has conducted a study of FT 
Resource Centers (Shive, Meieen> Harris> Vaughn and Grbgan 1980). Using case 
study mechbds Network researchers developed individual profiles of about 25 
resource centers which were then used to prepare a report on the resource 
centers as a whole • The profiles comprise the study's data base and they could 
be made available with some effort devoted to preserving the anonymity of the 
sites . 

Finally, the National Urban League is beginning a multi-year assessment of 
the social implications of FT. This study will examine the impact of FT in 12 
urban cities. It will look for indirect effects of FT such as increased community 
involvement in schooling as well as improved health and social services. The 
study IS just starting and data will not be available for many months. 

1 .3 Sxaittples of Questions that Can be Addressed by Extant Data 

This discussion will concentrate on relevant study questions that can be 
addressed by the data sets noted above. It focuses on questions related to the 
iraihediate effects of FT on children, parents and teachers, i.e. the effects of 
FT on program participants while they are in the program. It should be under- 
stood that this discussion (and other discussions of research questions) is not 
intended to provide a census of all possible study questions. Rather > examples 
which illustrate the range of i»tehtial research questions will be identified. 

1.3.1 Questions R i slevarit to the National Evaluatidh Data Base 

Several questions about the immediate effects of FT can be addressed by 
using the national evaluation data. There has been limited analysis and even 
more limited re analysis of the national evaluation data- The data base is hude 
and rich, containing multiple measures on multiple cohorts of children, parents 




and teachers from grades K-3 . In ho way were the AAI researchers abie to fully 
explore the depths of this data set. Examples of questions that have thus far 
gone uriaddressed or have not been addressed fully include: 

• What is the relationship between amount of FT expo siire and 
achievement? How does this relationship vary by model? 

• What effect does FT have oh school attendance? Hdw does 
this relationship vary by model? 

The national evaluation concentrated on the estimation of child effects 

rather than on a thorough analysis of data from FT parents ahd/br teachers. 

Noting that these data were not fully expilbred in the national evaluation, Haney 

and Pennington (1978) conducted a descriptive analysis in order to learn more 

about FT as a comprehensive service program, to better understand FT in terms of 

social action and parent involvement, and to help interpret the results of tUe 

national evaluation. Questions that could be explored further in this area 

include : 

• What has been the role of FT teachers and teacher aides? 
How does the role differ by model? 

It is also important for researchers and metaevaluatbrs to reanalyze the 

data set used by Stebbins et al . to see if the national evaluation findings 

and cbhclusibhs hold up under close scrutiny. House, Glass, Mclean and Walker 

(1978) authored the most frequently cited critique of the natibhal evaluation. 

Yet, in spite of their objections to the analysis strategy ased in the national 

evaluation. House et ai . did nbt reanalyze the raw child level data, but chose 

instead to accept AAI ' s site level results and based their "reanalysis" on the 

aggregate site level data presented in various national evaluation repbrts 

(Stebbins a*: al. 1977; Bock, Stebbins and Proper 1977; associated appendices). 

Thus, regardless of their -tomplaihts abbut the analysis strategy. House et al . 

by definition accepted the basic analysis techniques employed by the original 
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evaluatbrs. The same criticism can be ie>7eied at other FT met aevaiuators ie.g. 
Bereiter and Kurlarid 1978) . 

Thas, there is work to be done with the national evaluation data base 
simply in terms of checking the ftAI analyses for reliability: 

• How reliable are the national evaluation results? E>o 
they stand up under the scrut iny of secondary analys is? 

• Do differ ent researchers using differ ent analys is methods 
making different assumptions at r ive at different cbnclusibris 
about FT? About individual FT models? 

To this writer's knowledge, eamiiii's (198Q) doctoral thesis presents the 

billy child level reanalysis of the national evaluation data that has been done to 

* date. Among other analyses, Caihilli tried several solutibris to the problem of 

covar lance adjustment, explored the generali 2 ability of the national evaluation 

model and instrument categories; and examined the relationships of parent csnd 

teacher variables to outcome gains. According to Camilli, his "...reanalysis 

does not produce shocking discrepancies..." with the national evaluation findings. 

1.3.2 Questions Relevant to S ponsor and S ite Data Ba s e ^^ 

Other sources of information; in addition to the national evaluation; are 
relevant to FT. These include, for example, sponsor and site data sets. Many 
of the questions about model effectiveness addressed in the national evaluation 
can be addresised using spbrisbr data. However, given the diversity in measures 
and designs; model comparisons or "horse races" would not be possible except at 
the most global level: 

• What are the effects of participation in a given FT 
model on achievement? On affect? On attendance? 

• What can be said about causal relationships among 
these outcomes? 
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NIE/FT, B1 1 , 12/18/80 

The sponsor data sets also contain unique information such as Kansas' data 

on consumer satisfaction with the program. In this case, ah annual survey of 

program children, parents, teachers and administrators is used to gather data on 

perceptions about the prograrhr level of sat isf act ibh ^ arid suggestions for 
imprbvemeht . 

• How satisfied are FT consumers (parents, teachers, children)? 

• What is the relationship between parent satisfaction and 
child achievement? And attendance? 

Another interesting and ne looted area of work could cdnceritrate on agree- 
ments and disagreements a'bbut FT when viewed from the perspectives of different 
data sets. Clearly, evidence apart from the national evaluation shculd be 
integrated into an overall picture of FT: 

• Do different FT data sets (national evaluation, spbrisdr , site) 
lead to different conclusions about the success of FT models? 
if so, why? 

Finally, rib data were ever collected in the national evaluation as to the 
degree to which the FT models were implemented. Questions as to the conformity 
of the treatments to sponsors' specifications or even as to the existarice bf the 
treatments were never addressed systematically. Some sporisbrs have since invested 
substantial resourced in documeritirig the implementation of their programs and 
some of their implementation data sets are worthy of analysis. 

• How have FT sponsors gone about studying the implementation of 
their programs? Do their methods have implicatibris fbr the study 
of iJtipiementation in other programs? 

• Do the data that FT sporisbrs have collected on program impie- 
meritatibri allbw distinctions to be made among FT models? bo 
the data accurately describe what sponsors think should be 
happening in their classrooms? 

• ay their own analysis are FT _ sponsors able tb implement their 
mcxiels? Does external reanalysis cohfirin their cbriclUsibris? 
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2 . 0 Whait-Cah We Learn from Corhpilihg and Analyzing Follow~tjp Data on 
FT Partxcipanti ^ "~ 

The possibility of PT models having delayed effects (sleeper effects, 

long-term effects) has been raised at several points over the past decade. The 

original purpose of FT was to improve the life chances of children from low 

income families. From this perspective it makes sense to devote attention to 

post-PT effects, especially consider ihg that concern over the apparent fade^ut 

of Head Start effects (Wolff and Stein 1966) was the primary impetus behind the 

establishment of PT . However, as Goodrich and St. Pierre (1979) note "Studies of 

pDst-treatment effects have been seen as interesting but not timely.-." This 

section of the paper addresses the question of studying delayed FT effects. 

2. 1 Mot i vat i on foe CondQcting Follow-up Studi o 

In spite of relevant planning work by Good:;ich and St. Pierre (1979), Caret 
(1977), Madaus and Marx (1973) and a proposal by the School District of Phila- 
delphia (1980), studies of delayed FT effects have not been funded at the 
federal level. Several reasons for studying pbst-PT effects are given in these 
sources and are summarized here. First, FT was initiated because the achievement 
gains obtained by Head Start children appeared to dissipate in the early school 
years . The same issue is bf concern to FT-- if achievement gains are produced by 
PT, do they dissipate, or are they sustained after cbmpletibh of the program? 

Second, many FT sponsors try to impact childrehs' affect, conceptual 
ability, cognitive style, other non-achievement areas and parental behavior; 
These sponsors believe that achievement gains will be obtained indirectly 
through impacts oh mediating variables such as those listed above. In this 
case, effects oh student achievement are not expected to occur quickly. In 
order to evaluate these sponsors* models it is necessary to examine achievement 
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gains subsequent to prdgram teirmiriat ion . The question is, do FT mckaels that aim 
for achievement gains by first impacting mediating variables produce delayed or 
"sleeper" effects? 

A third motivation for studies of post^FT effects is that FT projects, 
by design or as a byproduct r inay affect p>arerital arid cbmmuriity opinions. Iri 
generalr parents support TT and have been a potent force in maintaining the 
program m the face of efforts to phase it out. The question which might 
arise is, what are parents' , teachers* and students* opinions of FT once 
students have made the trarisitiori to "regular" public school classes? 

Fourth, FT's overall objective has been to improve the "life chances" of 
children from low income families. College attendance, future earnings* social 
mobility and other post-school variables are all important indicators of success 
in life but are not available measures iri ariy existirig pdst-fT data base* Life 
chances studiiss could concentrate on earlier, more accessible indicators of 
success such as school attendance, special education placement, grade retention, 
dropout rate, grades, course selection, and discipline records. 

Although post- FT? studies have not been funded at the federal level , iriterest 
has beeri high ambrig spbrisdrs where debates about "sleeper" effects and fade-oat 
have taken place regularly. Studies of varying degrees of sophistication have 
been conducted by the following sponsors: Arizona (Riley 1978; Cloud r Rentfrow, 
and Hildebrandt 1979), Bank Street (Seitz, Apf el , arid Efron 1977), Far West Labs 
(Edwards and Bridewell 1979) , Karisas (assorted reports) , and Oregon (Weber and 
Fuhrmariri 1978; Becker arid Eriglemann 1978). 

2 . 2 What Follow-Up Data are Available? 

It is clear that some sponsors have relevant data. Based bri their survey 
of spbnsor data sets, Goodrich and St. Pierre (1979) recbitimerided six data sets 
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as being appropriate for producing new knowledge about pDst-FT effects, three 
of the data bases, those supported by Philadelphia, Fbrdhani and Oregon, afford 
especially gbcd opportunities since they meet fairly stringent methodolDgical 
standards (ample sample size, several sites, a wide range of measures # good 
local comparison groups). The other three data sets, maintained by Arizona, 
Georgia State arid SEDL, would support smaller studies in single sites that are 
somewhat weaker methbdologicaily . It should be noted that while other sponsor 
data sets were hot recommended for study by Goodrich and St. Pierre, several FT 
sponsors have upgraded their data management systems during the past year or 
two, and new data bases may be available. 

2.3 Sxamples of Questions that Can Be Add ressed by ?dlldw-Up Studies 

Table 5 presents study questions of particular interest in this area that 
could be addressed by the best of the sponsor and site supported data bases. 
In spite of the high quality of some spbhsbr/site data bases, they tend to 
concentrate oh paper and pencil butcbrhe measures, and are relatively weak on 
"life chances" behavioral variables (e.g. grade retention, special education 
placement, attendance). Further, none contain foilow-up data on the perceptions 
of children, parents and teachers about FT or data on parents' or teachers' 
educat iohal/eraplbyiheht status. 

There is ah important precedent for collecting long-term follow-up data on 
the school behavior of children- The Developmental Continuity Cbhsbrtium 
(£azar, Hubbell, Murray, Rbsche, and Royce 1977) analyzed fbllow-up data that 
were oDllected from children aged nine to 19 who participated in 14 different 
experimental infant arid preschool programs. Their study shows positive effects 
of Head Start on special education assignment and retention in grade (program 
children were assigned to special education classes and retained in grade less 
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Table 5* 

acampIgS oi a«seacch Questtona thae Can bq Addceaaed by 
Selected Sponsoc/Site Data Baaea 



RBSEARCB QUBSTICfIS 


SELECTED DATA BASES 




Phiia- Georgia 

delphia Fordham Oregon Arizona State SEDL 


m What are tbe causAl relatXbaaliXpi awrig acadeaic acb£even«nt> 

IQ> pccceis aeaiucei (ajuunt 6t B*f expoauce, attendance* etc.), ~ 
1 and affective aeaaucea? 


XXX 


Do acblevexnent efgecta aaaoclatod with pact icipat ion in FT in-' 
cceaae, decceaM* oc stabilize aftec completion oil the pcoqcaa? 

- It poiitlve effects bit acad^aic aehiev^eent aite pf oduced by 
FT liibdela at the end of grade Ctieee, do Ehey fade'-out, Qt 
are they auatalhed la the pdat'-FT years? 


X ; X X X 


* Do FT models that aia £or achievement gaintf by first changing 
children's affect* conceptual ability* parental behavior* and 
so on* produce 'sleeper effecta* on achiewaient measures? 


X 


^ Uow_does post^FT acadenic performance celate to previous FT 

pretest, denbgraphic data)? ^e gains suatained in ^Ha post- 
ET years? 

- - - -- - - - - 


XX XXX 


« 'rfhat are the relative Importances of the current acodeaiic year 
and the child's previous acadenic history as determinants of 
academic success? 

... _. . _ 


XXX 


e What Is the causal relationship betveeQ_^sdettic performance 
arid school attehdahce? In FT? Sfter FT? 


XXX 


• What Is the relationship between poat->FT academic performance 
measures and length of FT exposure? 


XX X 


• I^.i-t possible to predict pbst~PT patterns J fade-out, sleeper 
effects^ sustained growth^ etc. } on different measures in 
terns of the FT curriculum? 


X X 


• What differences £h post -PT -per fbcflshce (achievement, grades, 
attendance) exist Mbhg different models the sane stEe? 


X 



'Adapted £rom Goodrich ard St. Pier re (1979) 
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often than comparison children) . They also fbuhd evidence for fade-out of IQ 
gains over a three-year period. 

Whxie the findings of this study have been controversial, Cloud et ai . 
(1979) investigated the later schov-)! success of children enrolled in the FT 
program in Lakewood/ New Jersey and were able to replicate the findings of 
I^azar et al . Children who participated in FT were fdtand in high school to have 
Significantly lower rates of grade retention and special education placement 
than comparison groups composed of older siblings. The groups did not differ 
significantly on variables such as dropout rate/ educational achievement/ or 
educational aspirations • 

Such work suggests strongly that further research is needed in this area* 
Discussions with FT sponsors and sites (e.g. Philadelphia) lead to the conclusion 
that ih some instances it would be possible to obtain child level life chances 
variables from school districts records and append them to existing sponsor/ 
Site data sets. If augmented in this fashion some of the sponsor/site data 
sets would allow researchers to address questions such as: 

• What is the . long-term impact of a given FT model on life 



chances variables? 



• Do the results of such studies replicate those of tazar 
et all . and Cloud et al . ? 
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3 .0 What Can We Learn from New Studies of Sponsors and Sites in FT's Current 
Framework ? 

The viewpoint set forth earlier, that there is more to be learned from FT, 
holds as much for completely hew studies as for work based oh extant data • How- 
ever , a review of the use of FT research funds since completion of the national 
evaluation in 1977 shows that it may be easier to generate than to implement 
ideas for new research. 

Table 6 suinmarizes the work that: has been funded with FT research monies 
from fiscal year 1977 through fiscal year 1980 — more or less since completion 
of the national evaluation. Inspection of this table shows that in the past few 
years there has been a problem spending FT's research funds. This is due to 
factors such as the fact that the admihis tratibri has been trying to phase but 
the prdgrain for years . Instability caused by uncertainty about continuation of 
funding has lead to great problems in planning a coherent research agenda. In 
addition, the Division of Follow Through and the Office of Prograin Evaluation 
have consistently had conflicting bpihibhs as to the most appropriate research 
and evaluation activities that should be undertaken. Finally, the conduct of FT 
research has been complicated by external factors such as the Congressionally 
imposed cap on spending for consultant services* In 1980 this cap forced the 
Office of Program Evaluation to return to the U. S. Treasury $2 mil lion that 
would otherwise have been spent on FT research; 

In spite of these structural and organizational constraints there is no 
dearth of ideas for new FT research, and a good deal of work was contracted 
between 1977 and 1980. Consideration of the substance of the studies listed in 
Table 6 reveals that it 2.s possible to discern at least three broad thrusts in 
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Table 6 

rr Regearch Exp^naXtures (1977^1980)* 







contract 




Year 


Cbht:racrbr 


Xffibuht 


Description of Contract 


1977 


AAX 


552,512 


Cpmpletipn at national evaluation 






35/762 


Completion of .national, cost analysis 




SKI 


53,635 


Maintenance b£ na:tibnal data base 




Huron 


21/939 


Planning new studies 




Far West Labfi 


4/500 


Paper on FT implementation 


1978 


AAI 


471 /5 1 1 


Plamiihg for later effect;^ studCxes^ seareti 








for new models 




_..3RI 


45 / 490 


Maintenance of national data base 




Network 


597 ,618 


Muiti'*prograJtt study of dissemination 








strategies 




sbc 


3Sd / Odd 


Muiti** program study of parental invblveneht 




Huxon 


17,268 


Parent/teacher reanalysis 




JUidEiab 


SO, 769 


Study of FT supplemental training 




Hi 9H/ S^b pe 


79; 278 


Study of . potential for a joint FT modetl 












Oregon 


60,052 


Study of implementation in a new site 




_ Georgia 


53/682 


Planning paper for new FT research 




Georgia State 


45; 352 


Planhihg paper for new FT research 




Kansas 


70/739 


Planning paper for hew FT research 




Norrh Carolina 


48/864 


Plarxning paper for new FT research 




Oregon 


49,799 


Planning pap«r for new FT research 






48/592 


Planning paper for new FT research 


1979 


SDC 


450,000 


Multi-program study of parental involvement 




SRI 


14,825 


Archive national data base 




Philadelphia 


9/999 


Build data^base for delayed effects study 




RetiaB 


12;730 


Study of FT supplemental trsining 




AMS 


300 /Odd 


beveibp performance indicators and systems 








'o?^_?iO'^^9^4"g_P*?^??^*'5ce of FT sites 




Oregon 


143/418 


Study of _ implementation in a new site 




NortS Carolina 


28;911 


Data analysis 




Boon-^Young 


295/321 


Design of studies to be cbhdticted by pilot 



projects including extension of ^__P^yond 
grade 3, effects. of alternative levels of FT 
services; capabilities of WSXs 



1980 National Urban 987/279 Assessment of _ social implications of FT 

League in urban cities 



■AL BY ySA R 





Total 


Total 




Year 


Sbeht 


Allbcated 


Allbcated-'Speht 


19?7 


$168/398 


$1/000,000 


$831/602 


1978 


1,989,014 


2/OQO/OOD 


10/986 


1979 


1,255,204 


i;273,000 


17,796,,, 


1960 


987,279 


3/ddd /Odd 


2/012/721 



The data for this table were supplied by Jerry sums of SD/OPE. 

>* _ _ _ _ _ __ _ _~ 

This reflects the tbtail contracted fund*. Other small ambiihts of FT 

research fxxnds were spent for items such as conferences ahd travel. 

* S2 million xretumed to the U. S. Treasury because of Concreaaionally 
imposed cKp on asrisultaht services: 
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recent FT research: (1) planning new FT research; (2) imprbvihg FT rhahagemeht 
and implementation; and (3) new FT research (see Table 7). 

Since this paper focuses on information to Be learned from new FT studies > 
we will consider further s€5lected planning studies shown in Table 7 as well as 
other less formal efforts. In order to benefit from the thinking of others ^ the 
next section presents a review of several proposals for new FT research. 

3 . 1 Review of Research Proposals from FT Sponsors 

Many FT sponsors have been active in attempting to initiate hew FT research - 
In 1978 USOE awarded contracts to six sponsors , in part to elicit their ideas 
about potential new FT studies* ft review of these and other related reports by 
FT sponsors yields the following information. 

3.1.1 University of Gebrg^ia (Sllett, Hawrj, Pool arid. Sir.bcic 1979} 

This report addresses issues related to a second national FT experiment 
centering on child outcomes. The authors take the point of view that' redoing 
a large-scale multi-year national study is wori:hwhile it can be improved 
through measuring a variety of program outcomes arid through gathering implemen- 
tation and process data that" would allow explanations of how model-specific 
outcomes were achieved. Research to be undertaken through such a national study 
would address questions such as: 

• What is the relative effectiveness of FT models on a 
cbminbh academic perfbrmahce measure? 

• What is the relationship between level of program 
implementation and outcomes? 

• What are the contributions of individual model processes 
to common learner outcomes? To outcomes unique to each 



model? 



23 




Table 7 

FT Research Studies (1977-1980) by Type of S4udy 



Year 


Type of Study 


Planning New Improving FT Management 

FT Research and ImplementatJ.f^n - New FT Research 


1977 


• Huron planning • Far West Xiabs 
study implementation 

paper 


1978 


• AAI delayed • Oregon implementa- • Network dissemina- 
effects, new tioh study tion study 
models study 

# High/Scope, Kansas • SDC parent involve- 

• Sponsor joint modejL study ment study 
planning 

studies • Rehab supplemental 

training study 


1979 


. a Philadelphia • AMS development # SDC parent invoive- 
del£./ed effects of performance ment study 
study indicators 

• Rehab supplemental 
• Boon-Young • Oregon implementa- training study 
design of new tion study 
studies 


1980 


• National Urban 
League social 
imp! icat ion s 
study 
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Perhaps the major problem raised in the paper relates to measurement. What 
sponsor -spec 1 f ic > implement atibri arid process measures should be used in a national 
evaluation? If such a study were to be funded it would heed to be preceded by a 
period of measure development, perhaps extending the work begun by several FT 
sponsors (e;g; the Productive Language Achievement Test, the BRACT, several 
attempts to develop indices of implementation) . 

3.1.2 Georgia State (Hedges, Carter, Cooper, Surge and Myhatt 1979) 

Hodges et al . (1979) work under the premise that the principal vehicle for 
future FT research will be a large scale multi-year study involving many spoaGors 
and sites. They hope to improve on the national evaluation by offering a 
complex design that involves implementation of different FT models in the same 
cbihihuriity, and by broadening the measurement battery to include dutcdmes (crdss- 
spdnsdr and spdnsdr -specif id ) , prdcess , and implementation measures; The types 
of questions to be addressed through the studies suggested in this report 
include : 

b What is the effect of PT models oh classroom practices? 

• What happens to classroom practices when the spdnsdr is remdved? 

• What is the effect of model implementation on child outcomes? 

• Are models differentially effective? 

• What is the effect on implementation and outcomes of spc)nsorship_ 
(the typical FT arrangement) vs. knowledge provision (the typical 
NDN or PIP arrangement)? 

3;l-3 Dmversity of Kansas -(J^amp and Stivers 1979) 

Ramp and Stivers suggest that future FT research be focused in two general 
areas: (1) The relationships between educational processes and products; and 
(2) sponsors as social change agents. The first focus raises quest idhs abdut 
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what philosophies of education work "best", what philosophies of education 
produce what effects, and what classroom events (across models) produce what 
child outcomes. A focus on sponsors as change agents raises questions about 
which sponsor activities ensure the most effective delivery of a program, and 
what must be known about a commuhity/schbol prior to implementation of an 
innovative program. In addition. Ramp and Stivers call for the development of 
sponsor -spec if ic measures of model implementation • Relevant research questions 
include : 

• What are the relative effects of FT models on cbmmdh outcome 
measures? 

• What are the effects of individual models on sponsor-specif ic 
measures? 

• What are the components of an effective model delivery system? 

• What are the initial site features that effect implemehtat ion? 

(i What is the relationship between site characteristics and 
degree oL implement at ion? 

• What are the effects of varying the number of adults in the 
classroom on achievement, program costs, consumer satisfaction? 

3.1.4 North Carolina (Olmsted 1979) 

The approach of this paper is that a series of small, intensive, well 
designed studies will yield more and better information than a single large 
study. Measurement would be intensive and would focus on outcome measures 
derived from each of ET's components including cognitive and affective develop- 
ment, parent involvement, staff development, and comprehensive services. In 
addition, measures of model implemehtatio;; would be crucial. Such small 
studies would address comparisons both among and within sponsors and would be 
integrated via meta-analysis techniques (Glass 1977) . Several specific studies 
are proposed to answer questions such as: 
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• What are the differential effects of FT modeis on seif-concept / 
iocus of control^ and achievement? 

• W_hat_is_the relationship between the home environment and 
child affective and achievement outcbmes? 

©Are jshdrt-term gains in achievement suistained fdllbwihg 
graduation from FT? 

• What is the impact of FT's comprehensive services in terms 
of degree of utilization^ client sat isf act ion> and child 
health status? 

This paper also proposes that a complementary effort be undertaken to conduct 
a national survey of FT schools and sponsors, concentrating on documentation of 
model implementation and local site characteristics. No outcome measurement 
would be done as this survey would provide descriptive data useful for program 
mahagemeht and for the interpretation of results from the smaller evaluations. 

3.1.5 University of Oriagon (Becker et a l. 1979, C a rnine 197 Bj- 

The point of view offered in these papers is that continuing research 
should be conducted within FT in order to demonstrate "what can be done to 
improve problem areas where schools currently fail the most." Dembhstrat ion 
projects are called for in the fbllbwihg arisGs: 

• A large study with improved performance measures, assessment 
of implementation, and better design. 

• A study of the effects of extending FT to grades 4, 5 and 6. 

• A study of planned variation in approaches to bilingual 
educat io n . 

• A study of sponsor effectiveness in large cities. 

In addition, studies are also recommended to address the following questions: 

• What are the effects of FT curricula independent of management 
systems? For example, does the curriculum (e.g. DISTAR vs. a 
basal reading program) make a difference when holding the 
management system (e.g. Direct Instruction sponsorship) constant? 
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• What aire the effects of FT spbrisdrship? For example, compare 
the effects^of a FT model curriculum as implemented with arid 
without a sponsor . 

• What are the effects of different components of the sponsor's 
management systems in terms of increasing learning opportunities 
for childr en? 

• What are the effects of various types of iJarerital involyemeht? 
What are the effects of models with arid without parerital educ- 
ation components? 

• What are the independent effects of different subject area 
components on student outcomes? 

• What are the effects of different levels of funding on program 
outcomes? 

• What xs the impact bri fidelity of treatment arid student per- 
formarice of gradually reducirig the spdrisdr's irivdlvemerit in 
a site? 



• What IS the effect of management strength by project officers on 
contract compliance by sponsors and sites? 

• What is the effect of varying academic engaged time? 



3.1.6 Southwest Ed uc at ibrial Develbpmerit Labor atbry (Lumbley arid Krbrikbsky 1979) 



Eike some of the other sponsor reports this paper assames a new, maiti-year 



national HTT experiment. It offers alternatives for accomplishing each step of 
the design and execution of such a study. Questions specific to SEDL's bilingual 
PT model are posed, e.g.: 

• What effect on child butcbmes cari be expected from differ i rig 
bilingual program emphases (maintenance vs. transition vs. 
developmental)? 

• How can the transition out of FT be made as smooth as 
possible? 

Further, the paper calls for a study which will allow collection of data that 
at e cbmpar able across spbrisors , arid also reflective of individual spbrisbr gbals^ 



Finally, a thorough process evaluation is recommended in order to document the 
sponsor's delivery system and the characteristics of implementation in each 
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site; provide for formative feedback; and collect implementation and butcditie 
dat a . 

3.1.7 FT Sponsor Evaluation/Research Committee (Gennari 1978) 

A task force of sponsor representatives was formed to consider the nature 
and type of future evaluation/research ac tivitieil for FT. A preliminary report 
from this group was issued late in 1978 and made many general recommendations 
regarding ways in which an appropriate atmosphere could be created for future FT 
research. For example > the group recommended strongly that future FT research 
be "multidimensional" (i.e. it should consider all of FT's program components); 
that the diversity of FT approaches should be respected through spbrisdr specifica- 
tion of critical and unique characteristics of their programs; that evaluations 
should focus on the "documentation of program changes ais they occur across a 
variety of educational contexts"; that future research studies be selected and 
funded oh the basis of input from all program stakeholders; and that a program 
wide ongoing data retrieval system be initiated to facilitate research. In this 
paper the groap did not offer suggestions for specific research/evaluation 
studies • 

3.1.8 Summa^ y b f FT Spo h s b r Rese^ aii^ ropbsal s 

After reviewing a few of the sponsor papers some patterns become clear . 
With one exception, ail of the sponsors who completed the planning studies appear 
to accept the notion of national FT studies which involve, in part, comparisons 
of sponsor effectiveness oh common achievement measures. The spkDrisbrs also note 
quickly that such natibnal studies make sense only in the light of improved 
research design, measurement of sponsor -specific outcomes, and measurement of 
implementation and process variables. These sponsors feel that the basic idea 
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of comparing the effectiveness of aitecnative educational models makes sense > 

we 3 list heed to do it better and with a more global orientation. 

On the other hand, many of the sponsors also call for a series of mote 

diverse studies of snaller scale to address a variety of research questions; 

Some of the most interesting of these deal with the delayed effects of FT 

mcdelSr the effects of iridividual components of the FT program/ the effects of 

varying academic liSarhihg time, the effects of ihstitut idnalizat ion of a sponsor 

program (withdrawal of the sponsor), and the effects of sponsorship. 

3 . 2 Review of Research Proposals from Other Planning Efforts 

In addition to the FT sponsors, many other planning efforts have been 
Undertaken- Selected efforts hot reviewed earlier in this paper will be 
sunmiarized here. These include papers by the Division of Follow Through , 
Boon-Young , and the Huron Institute. 

3.2.1 Division of Follow Through (1978) 

A short paper issued by the Division of FT in 1978 set forth a "blueprint" 
for studies that could funded in future years. These include research on: 

• The impact of non instructional services. 

• Spinoff effects on comihuhity, schools r teachers r a.*d parents. 

• Delayed effects. 

• The effects of preschool experience on FT children. 

• The development of new tests and other instrumentation. 

• The expansion of FT into grades 4*6. 

• The expahs ion of "sroall" sponsors that were not part of the 
nat ibr.al evaluation . 

• The relationship between parent Involvement and child 
outcomes i 
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• The effects of FT on special populations. 



• The startup of current sponsors in new sites - 

• The startup of new sponsors - 

• The utility of resource centers as a vehicle 
fdir dijgseniinating research strategies. 

Information in several of these areas has already been collected via the 

Network's study of resource centers, sbc's study of parent involvement/ the 

National Urban League's study of social change, Oregon's implemehtatioh study/ 

and AAI's search for new sponsors. 

3-2.2 Boon^'Young Planning Study (forthcoming) 

Motivated in part by the feeling that the national evaluation was too large 

to be manageable , this planning study was cbmmissidned by the Division of FT in 

the belief that the potential for acquiring useful knowledge is enhanced by 

using a small number of sites in each of a number of small scale research efforts. 

The purposes of the procurement were to provide planning, technical assistance, 

and design support in three areas: 

d A study/ experiment to deteririine the effects of extending 
FT be.yond grade three. 

o ft study/experiment to determine the effects of alternative 
levels of FT services within and among selected FT models. 

o Planning assistance to determine the capability of LEAs to 
design, implement/ and evaluate their own FT models. 

This study is due to be completed in February 1981 . Reports should include 

alternative srudy designs, feasibility analyses > cost analyses, and so on in 

each of the three areas. 

3.2.3 Huron -In stitute Plahhinq Study — ^aret aS77 ) 

In 1976 the Hurdn Institute was requested to identify FT evaluation research 
studies that might be pursued. A set of 25 research areas was identified, and 10 
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plans were described in some detail; Included were plans for: 

• Comparative analyses of sponsor and national evaluation data 
to determine ways in which sponsor evaluations differed from 
the national evaluation, factors related to differences/ and 
ways to strengthen FT evaluations. 

• Document atibh and field testing of sponsor -developed instru- 
ments in order to pull together the efforts of sponsors at 
assessing their specific objectives. 

• Establishing criteria for identifying successful models, 

• Determining whether appropriate achievement measures (perhaps 
criterion referenced) are available. to measure, children 's 
growth over time. Conducting a small scale pilot study. 

• Examination of the delivery of cbmprehehsive services in 
selected FT sites in order to identify effective strategies 
for service delivery and coordination. 

• Exploration and comparison of the forms of parent involvement 

in FT m an effort to assess the efficacy of various strategies. 

• Documentation and exploration of the process of model development, 

• Investigation of the relationship between "contact time" arid 
achievement . 



• Exploration of explanations for within-sponsor site to site 
variation in child achievement. 

• Assessment of the feasibility of using "site inspectors", e.g. 
site visitors , interviewers , participant-observers, in an 
evaluation of ET?. 



4 .0 Sumtnarjg and Recommendations 

TJie thesis of this paper is that there is^ more to be learned frcxn FT in it 
current framework. Itiree ways of generating new knowledge front FT have been 
discussed: (1) secondary analysis of extant data; (2) compilation and analysis 
of follow-up data; and (3) completely new studies. Although some of the ideas 
reviewed m this paper may be infeasible or iiroract icai , a case has been made 
that there are many areas in which we can learn from FT, and in which FT can 
demonstrate effective ways of educating ybUrig children. 
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while many of the research questiohs/prbpbsals reviewed in this paper have 
meir'-t/ two are recbinihehded in this cbhcludihg section for special consideration. 
The first of these is research oh the potential delayed effects of FT. There is 
an abundance of reasons to invest resources in this area- First, the long-term 
effects of educational programs is an important area of research. Of what 
purpose is education if there is no prospect for lasting effects? Second, there 
are ample precedents for studying delayed effects (e.g. Lazar et al . 1977); 
precedents which have had enormous impact on Head Start. Third, impact on "life 
chances" variables is the ultimate goal of the FT program and should be investi- 
gated if at all possible. Fourth, many FT sponsors hypothesize differential 
patterns of long-term effects and are in favor of such a study. Fifth, the data 
to perform delayed effects studies exist in a reasonable (though not completely 
adequate) form. Finally, since studies of post-FT effects would be secondary 
analytic in nature they would be relatively inexpensive and could provide 
information m a timely manner. Results from ah investigation into the delayed 
effects of FT on children could well have important implications for the design 
and funding of FT as well as other early childhood education programs. 

belayed effects studies could vail be expanded to address the long-term 
effects of FT on parents and teachers. This is an area of research that has 
received much less attention than studies of delayed effects on children. 
Examples of relevant questions that would entaiil new data collection include: 

• What are the long-term perceptions _of children and parents 
about FT? Do they remember FT? If so, in what respocts? 

• What are the impacts of FT oh parent level life chances 

variables (e.g. continued education/ employment)? Did FT 
lead parents to further their education? To gain new 
employment skills? 

• What are the impacts of FT on teacher l^>vel lii-? chances 
variables? Did FT provide useful training for teachers? 
Did it further their employment status? 
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A second area of research also merits further discussions Some of the 
sponsor research papers venture the notion thai, we should study sponsorship 
itself. This is an important idea that could lead to the integration of FT 
studies With studies of other mechanisms for change in schooling. The concept 
and operationalization of sponsorship is perhaps FT's most creative contribution 
to educational change. Yet< FT research has been so focused on child outcomes 
that very little effort has been devoted to understanding the key concept of 
sponsorship, defining it, or comparing it with other mechanisms for change. 

An investigation of sponsorship might begin with a comparative analysis 
of sponsorship vs. other change mechanisms (e.g. the National Diffusion Network, 
or Project Infdrtnation Packages). While we might equate PIPs with the simple 
provision of information^ and the NDN with the provision of information plus 
limited technical assistance, sponsorship entails (at the very least) the 
concept of a long-term relationship between the sponsor and an OS?^ as well as 
the introduction of a new curriculum or use of different teaching methods. The 
fact that a FT sponsor is committed to his/her model, and remains in a site for 
many years to help implement the model is a radical departure from other change 
i?trategies . 

Of key interest to a preliminary descriptive ajialysis would be questions 
such as: What assumptions about the educational change process do different 
change mechanisms make? What are the differences in type of services provided? 
in the intensity of services provided? Given such a descriptive analysis it 
would be possible to proceed with selected studies of the effects of different 
change mechanisms. For example, what is the effect of a curriculum as imple- 
mented under FT sponsorship vs. under some other change mechanism? What does 
the curriculum look like under each implemehtatidn strategy? What are the 
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respective outcomes? On the other hand, one can look within FT and ask/ what is 
the effect of a given curriculum when implemented by different FT sponsors? 
Again, what does the implemented curriculum look like? What are the outcomes? 
Finally, it wouid be important to consider what happens to a program when 
sponsorship is withdrawn? Do programs which are implemented via sponsorship 
stay implemented longer than those implemented by some other change mechanism? 

In closing it must be stated that all the planning done to date, including 
the effort pat into this and the other papers prepared for NIE, will go for 
naught unless some of the organizational and structural constraiT*;ts that have 
hampered FT research for the past few years are removed. This does not mean 
that the program has to receive a guarantee of fuhding-^-'research can be planned 
under conditions of uncertainty. It is critical, however, to find a way to avoid 
the situation of the recent past \n which the Division of Follow Through and the 
Office of Program Evaluation were often at odds about the appropriate directions 
to take with respect to FT research. This could be accomplished by assigning 
responsibility to a single group that would have the authority to plan and 
execute FT research and evaluation, or by dividing responsibility among multiple 
groups each having their own budget {as currently appears to be the case with 
the Division of Foliow Through and NIE having separate agendas and budgets). 
The problem to avoid is that of having multiple groups in%^olved, each with veto 
power over the other's activities. If this continues to be the case the question 
will not be whether there is more to be learned from FT, but whether the con- 
straints placed on the planning process will allow anything to be learned. 
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